1. Field of the Invention
The present invention relates to an image processing apparatus, image processing method, and imaging apparatus to enable noise reduction in an image that has been imaged with an imaging device, for example.
2. Description of the Related Art
For example, with an imaging apparatus such as a digital still camera or the like, various types of methods have been provided heretofore as methods to reduce noise in an imaged image. For example, in Japanese Unexamined Patent Application Publication No. 2008-177801 is disclosed a method of noise reduction processing in the spatial direction within one imaged image (hereafter this noise reduction processing will be referred to as “intra-screen NR” (NR is an abbreviation of “Noise Reduction”)).
That is to say, in Japanese Unexamined Patent Application Publication No. 2008-177801 is disclosed the reduction or removal of noise components within one image of each of imaged images, by performing intra-screen NR processing using an ε filter or bilateral filter. This is primarily to remove optical shot noise of the imaging devices such as CCD (Charge Coupled Device) and CMOS (Complementary Metal Oxide Semiconductor). Therefore, the intra-screen NR processing is performed as to a RAW image from the imaging device.
With the intra-screen NR processing, post-processing pixel output for a pixel of interest is obtained by setting a window region surrounding the pixel of interest and performing processing for multiple pixels included in the window region.
The ε filter is a filter to perform noise reduction by leaving the edge portions and outputting an average value. That is to say, the ε filter calculates a absolute value of difference between the pixel value of the pixel of interest and the pixel values of the other pixels in the window region, selects the pixels having absolute values of difference that are at or below a predetermined threshold and executes average valuing processing, and the obtained average value is set as the pixel value of the pixel of interest.
Also, similar to the ε filter, the bilateral filter is a conditional average value filter. The bilateral filter performs average valuing processing using a weighted coefficient Wd that is dependent on the difference between the pixel values of the pixel of interest and the other pixels in the window region, and the weighted coefficient Ws that is dependent on the distance between the pixel of interest and the other pixels mentioned above.
That is to say, the post filter processing pixel value fi of a certain pixel of interest Pi (where i is the pixel position within one pixel screen) is expressed as shown in Expression 10 in FIG. 41.
In the case of using this bilateral filter, noise reduction can be made more smoothly while further retaining the edge portions, more so than in the case of using an ε filter, and improvements in image quality can be expected.
Methods of intra-screen NR processing may include not only the methods described above that use an ε filter or bilateral filter, but also methods that have a lower capability from the perspective of maintaining the edge portions but use a robust filter, or methods that use a medial filter.
Techniques of noise reduction of the imaged image are not only the above-described intra-screen NR, but methods to reduce noise in the temporal direction by adding multiple imaged images have also been proposed.
For example, in Japanese Unexamined Patent Application Publication No. 2001-86398 is disclosed a technique for sensor-less shaking correction for a still image. That is to say, in Japanese Unexamined Patent Application Publication No. 2001-86398 is disclosed a method to take multiple images with a high-speed shutter operation such that shaking does not occur, and considering the effects of shaking, layers low illuminance images that have been taken, to obtain one still image having high illuminance. This technique is based on an idea that if continuously taken images are layered, the noise that is random components is dispersed, whereby noise is reduced, and from the perspective of noise reduction is a technique similar to a frame NR (noise reduction) for moving images. The moving image frame NR method is a technique to layer the current frame and reference frame in real-time, and the current frame and reference frame are constantly layered with a 1:1 ratio.
On the other hand, in the case of still image noise reduction, the reference frame for one current frame normally becomes multiple frames. According to the still image noise reduction method, the higher the shutter speed the more the influence of shaking is reduced, and as the number of layers increase, greater noise reduction effects and greater sensitivity can be expected.
By layering multiple images, the method in Japanese Unexamined Patent Application Publication No. 2001-86398 for noise reduction is performed as to YC data (made up of luminance component Y and color component C) after gamma correction or the like is performed, not as to a RAW image from the image device.